This invention relates generally to a data storage system and more specifically to a data storage system for reducing the amount of memory storage required to store sets of correlated data.
Data logging and analysis systems and data storage and compression systems, find applications in many diverse fields such as the medical area including cellular tissue analysis (cytology), X-ray analysis, and Cat Scan. These systems also find use in the fields of oceanography, seismography, meteorology and satellite weather photo based forecasting, and various other systems. A major problem in these systems is the memory storage size requirement which occurs when multiple parameters, each having multiple resolution levels, are to be cross-correlated into sets and the accumulated occurrences of each set counted.
For example, the cross-correlation of two parameters, each having 256 possible levels of resolution, requires 65,536 count storage locations. Similarly, cross-correlating four such parameters requires over four billion memory count storage locations, and cross-correlating five such parameters requires over one trillion count storage locations. In general, the number of count locations for these systems is R.sup.P where R is the number of resolution levels for each parameter and P is the number of parameters to be correlated. This number must then be multiplied by the number of bits needed to store the maximum count for any such correlated set to yield the total size of the memory. These enormous memory requirements are well beyond the primary and secondary storage capability of many present day computer systems except for a few extremely expensive supercomputers which are not practical for general use.
Additionally, real time data logging and correlation mandates the useof fast primary storage means, so as to avoid loss of incoming data. One approach has been to deal with the parameter data in a linear form, not cross-correlated, where each occurrence of each parameter signal for each level of resolution is logged and counter. Thus, where each parameter signal has one of 256 resolution levels, a system correlating two parameters would require 512 storage locations having a word length sufficient to store the maximum count which could be expected for any parameter at any signal resolution level. However, to derive meaningful information from the stored data in linear form, the parameter data must be cross-correlated, albeit not in real time, which still requires large amounts of memory storage.